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PROBLEM  SOLVING  OF  MID-CAREER  ARMY  OFFICERS:  IDENTinCATION  OF  GENERAL 
AND  SPECIFIC  STRATEGIES 


EXECUTIVE  SUMMARY 


Research  Requirement; 

The  Army  needs  to  have  a  better  understanding  of  how  military  leaders  solve  problems  in 
uncertain  battlefield  situations.  Often  these  situations  are  complex,  characterized  by  the  familiarity  of  the 
problem  to  the  problem  solver,  the  stakes  involved  in  the  consequences,  time  pressure,  and  the  effort  that 
is  necessary  or  available  to  devote  to  the  problem.  Although  the  military  services  have  largely  relied  on 
exhaustive  analytic  comparisons  or  “6  step  models”  for  decision  making  procedures,  recent  studies  found 
that  these  methods  do  not  correspond  well  to  the  needs  of  complex  tactical  situations.  Shortcomings  of 
rigid  procedures  point  to  the  usefulness  of  approaches  that  correspond  to  how  people  at  various  skill 
levels  actually  solve  problems  in  dynamic  situations.  However,  research  has  yet  to  provide  details  about 
the  utilization  of  actual  naturalistic  strategies  to  develop  higher  levels  of  problem  solving  skills.  To 
effectively  train  the  skilled  use  of  naturalistic  strategies,  a  deeper  understanding  of  tactical  problem 
solving  is  needed.  Strategies  (persistent  ways  of  thinking  that  individuals  use  to  solve  problems)  first 
have  to  be  identified,  their  advantages  and  disadvantages  in  various  situations  characterized,  and  methods 
for  improving  them  developed. 

Procedure: 

Eighty  Army  officers  were  interviewed. separately  about  the  ways  that  they  approached  and 
solved  problems.  Participants  rated  how  often  they  used  four  approaches:  analytic  comparisons,  6-step 
procedures,  a  recognition  process,  and  a  method  that  favored  an  option  that  was  initially  promising.  They 
talked  aloud  about  what  they  were  thinking  while  solving  two  tactical  problem  scenarios  and  indicated 
which  strategies  were  used  to  reach  their  proposed  solutions.  For  example,  one  candidate  strategy  was 
Broke  the  problem  into  smaller  problems.  They  rated  the  importance  of  strategies  they  used  and  gave 
examples  of  how  each  important  strategy  was  used. 

Findings: 

Participants  used  more  than  one  approach  to  solve  the  problems.  Responses  suggested  that 
participants  developed  solutions  by  using  either  a  set  of  prescribed  steps  or  developing  a  likely  candidate 
option,  but  not  both.  Although  Army  training  advocates  prescribed  step-wise  methods,  participants’  time 
in  service  was  not  related  to  their  preferences  for  using  those  methods. 

The  familiarity  of  the  problem  and  the  effort  expended  were  also  related  to  participants’  selection 
of  approaches.  When  the  problem  was  more  familiar  (e.g.,  seizing  a  river  crossing)  participants  indicated 
that  they  relied  on  recognizing  a  solution  based  on  their  past  experience  with  similar  problems.  When  the 
problem  was  less  familiar  (e.g.,  a  hostage  rescue),  participants  relied  either  on  developing  a  promising 
course  of  action  or  on  a  procedure,  but  not  both.  Use  of  an  analytic  approach  was  associated  with  more 
effort  to  think  about  the  problem,  while  the  recognition  approach  was  associated  with  less  effort. 
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Proposed  courses  of  action  for  the  more  familiar  problem  were  associated  with  the  use  of 
different  strategies.  Four  courses  of  action  were  related  to  the  strategies  used:  hold  back,  retake  the 
assembly  area,  take  the  assembly  area  and  forward  ground,  or  bypass  the  original  assembly  area  phase 
and  focus  on  the  second  phase  (bridge).  Strategies  were  then  compared  across  solutions.  Fifteen 
strategies  were  identified  as  indicators  of  which  course  of  action  was  proposed.  Seven  strategies  were 
positive  indicators  for  all  four  solutions.  For  example,  people  who  focused  on  the  bridge  did  not  report 
using  the  strategy  Used  specific  and  precise  comparisons.  People  who  held  back  or  focused  on  the 
assembly  area  were  more  likely  to  report  using  specific  and  precise  comparisons  in  their  thinking.  On  the 
other  hand,  Used  the  solution  that  had  worked  most  often  in  the  past  was  used  by  those  who  proposed 
either  the  hold  back  or  bypass  solutions.  This  suggests  that  the  use  of  prior  knowledge  might  have  been 
more  of  a  factor  in  these  solutions  than  the  others  involving  the  assembly  area. 

Strategy  use  can  also  be  interpreted  based  on  the  results  of  other  research  on  problem  solving  in 
uncertain  situations  and  how  prior  experience  is  employed  by  more  and  less  skilled  participants.  For 
example,  solutions  that  included  moving  forward  to  the  bridge  were  more  likely  to  be  associated  with 
consideration  of  the  relevancy  of  available  information.  Based  on  research  of  expertise,  one  would 
expect  that  this  strategy  would  be  one  that  would  be  used  by  the  more  experienced  problem  solvers. 
Similarly,  those  who  proposed  the  “hold  back”  solution  and  the  “take  back  the  assembly  area”  solution 
were  less  likely  to  report  looking  for  flaws  in  the  plan  than  those  who  proposed  courses  of  action  that 
went  forward  to  the  bridge. 

Utilization  of  Findings: 

A  better  characterization  of  naturalistic  approaches  and  strategies  for  complex  tactical  scenarios 
will  result  from  identifying  how  military  leaders  approach  problems  and  what  strategies  they  employ  to 
plan  a  course  of  action.  By  developing  a  fuller  characterization  of  the  naturalistic  strategies,  their 
advantages  can  be  exploited  and  their  disadvantages  can  be  identified  and  fixed. 

These  interim  findings  illustrate  that  differences  in  thinking  processes  and  relationships  between 
strategies  and  solutions  influence  course  of  action  planning.  Ongoing  work  to  identify  differences 
between  strategies  used  by  the  less  and  more  skilled  problem  solvers  can  be  exploited  to  facilitate  training 
of  those  with  less  skill  or  fewer  experiences  to  employ. 
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PROBLEM  SOLVING  OF  MID-CAREER  ARMY  OFFICERS: 
IDENTIFICATION  OF  GENERAL  AND  SPECIFIC  STRATEGIES 


Introduction 

Despite  great  interest,  there  is  much  uncertainty  about  what  leads  to  skilled  problem  solving  and 
decision  making.  When  a  problem  solver  is  faced  with  the  task  of  making  a  high  quality  decision  in  a 
complex,  variable,  ill-defined,  and  novel  real-world  problem,  the  optimal  approach  and  a  distinct  goal  are 
seldom  obvious.  Time  is  often  short  and  the  solution  frequently  has  high  stakes  consequences. 
Nevertheless,  people  manage  to  solve  these  types  of  problems,  often  relying  on  their  prior  knowledge. 

How  people  manage  this  is  not  usually  clear,  although  different  approaches  have  been  identified. 
“Approach”  differs  from  “strategy”  primarily  in  the  level  of  processing  chosen  for  analysis.  Where 
"strategy"  is  a  regularity  in  reasoning  that  guides  thinking  in  a  particular  situation  (Bruner,  Goodnow,  & 
Austin,  1956),  an  "approach"  is  more  an  over-arching  combination  of  strategies.  Moreover,  in  real-world 
settings,  separation  of  problem  solving  from  decision  making  is  somewhat  arbitrary  (Connolly  & 

Wagner,  1988).  Thus,  no  effort  was  made  here  to  distinguish  problem  solving  from  decision  making 
strategies. 

Many  problem  solvers  appear  to  follow  a  natural  thought  process,  using  their  knowledge  and 
experience  to  understand  the  situation  and  to  determine  what  to  do  (Klein,  1989).  Often  the  situation  is 
recognized  as  familiar  and  a  past  solution  implemented.  For  less  familiar  situations,  workable  solutions 
can  be  developed  by  selecting  a  likely  candidate  and  developing  it,  by  using  strategies  such  as  breaking 
the  problem  into  smaller  problems,  by  changing  the  problem  into  a  different  type  of  problem,  or  by 
employing  analogies. 

Four  approaches  have  received  research  attention:  analytical,  procedural,  recognition,  and 
dominance  structuring.  These  four  vary  primarily  in  their  reliance  on  different  components  of  the 
problem  solving  process  to  ensure  effective  solutions.  Procedural  models  emphasize  the  importance  of  a 
sequential  framework.  Analytic  models  rely  on  thorough  assessment  of  information.  Recognition  models 
emphasize  one’s  use  of  prior  experience.  Dominance  models  hinge  on  structuring  the  process  using 
one’s  values  and  sensitivity  analysis.  Other  factors  such  as  time  available,  importance  of  the  outcome, 
and  the  effort  required  have  also  been  demonstrated  to  influence  strategy  use  (Payne,  Bettman,  & 

Johnson,  1993). 


Background 


Procedural  Approach 

Historically,  both  problem  solving  and  decision  making  have  been  described  by  a  step-wise 
approach.  With  variations,  this  is  basically  a  sequential  procedure  resembling  the  following:  define  and 
reorganize  the  problem,  identify  information,  list  possible  solutions,  test  possible  solutions,  select  and 
implement  the  best  solution  (e.g.,  Bransford  &  Stein,  1993;  Janis  &  Mann,  1977;  Miller,  Galanter,  & 
Pribram,  1960;  Newell  &  Simon,  1972). 

The  benefit  of  this  approach  is  that  it  provides  a  set  of  subtasks  that  can  be  applied  to  solve  a 
variety  of  problems.  Educators  often  develop  an  unfounded  belief  in  the  inherent  capability  of  the 
process  itself  to  identify  a  solution  and  that  skillful  use  of  these  steps  is  obvious  to  the  problem  solver. 
However,  the  procedure  gives  little  guidance  as  to  how  additional  information  might  modify  or  eliminate 


1 


the  steps,  or  how  they  can  be  supplemented.  Likewise,  "[t]he  learner  who  memorized  only  a  series  of 
steps  will  be  less  likely  to  know  what  knowledge  he  or  she  possesses  might  be  useful"  (Catrambone, 

1995,  p.  34).  This  would  be  an  obstacle  to  solving  a  problem  which  the  problem  solver  has  not 
encountered  before. 

Analytical  Approaches 

A  number  of  prescriptive  models  have  been  developed  as  techniques  to  aid  in  solution  selection 
(Anderson,  Deane,  Hammond,  &  McClelland,  1981).  These  analytical  methods  dictate  a  sequential 
procedure  to  identify  the  option  with  the  highest  utility  value:  define  the  problem,  identify  the  set  of 
options,  eliminate  unreasonable  alternatives,  list  all  comparable  attributes,  assign  utility  and  weight  values 
to  each  attribute,  calculate  the  total  for  each  alternative  across  its  attributes.  Utility  is  often  defined  as 
monetary  value  and  weight  as  importance.  The  alternative  with  the  highest  total  utility  value  is  chosen. 

These  techniques  provide  an  internally  consistent  logical  system  to  organize  the  information 
about  possible  options  and  their  attributes  so  that  an  optimal  solution  can  be  identified  and  selected.  The 
quality  of  the  outcome  depends  on  accurate  knowledge  of  all  options,  criteria,  and  values.  However, 
these  approaches  have  become  the  basis  for  much  decision  aiding  technology  by  showing  people  how  to 
make  their  decisions  conform  to  the  technique. 

Getzels  &  Csikszentmihalyi  (1976,  pg.  248)  note  that  "A  person  who  claims  his  ...  problem 
ultimately  [is]  solved  so  that  it  cannot  be  altered  any  more-must  believe  that  he  has  found  the  correct 
solution;  but  correct  solutions  exist  only  for  problems  that  have  already  been  solved  before."  In  complex, 
dynamic,  ill-defined  problems  that  are  novel  to  the  problem  solver,  the  indisputable  correctness  of 
solution  is  not  singularly  Important.  Rather,  what  is  important  is  how  workable  the  solution  is  up  fi’ont 
and  how  successful  it  was  after  the  fact.  Beach  and  Lipshitz  (1993)  point  out  that  analytical  models  are 
based  on  a  theory  that  "is  an  abstract  system  of  propositions  that  is  designed  to  describe  the  choices  of  an 
ideal  hypothetical  decision  maker— omniscient,  computationally  omnipotent  Economic  Man  (p.  21)." 

Thus,  in  real-world,  ill-defined,  dynamic  situations  where  all  the  information  is  not  known  nor  knowable 
and  when  time  is  not  available,  these  models  lose  their  usefulness.  For  example,  Payne  et  al.  (1993) 
found  that  performance  of  decision  makers  was  sensitive  to  time  pressure  and  the  requirement  for  effort 
or  accuracy. 

Dominance 

In  contrast,  Montgomery  (1989)  proposed  that  a  person  defines  the  relevant  dimensions  of  the 
problem,  and  identifies  a  promising  alternative  by  eliminating  any  alternatives  that  are  unattractive  on 
important  attributes.  An  alternative  is  then  immediately  chosen  if  it  is  better  than  all  other  options  on  at 
least  one  dimension  and  equal  to  the  other  options  on  all  the  other  dimensions,  or  on  some  absolute 
criteria.  However,  if  the  promising  alternative  does  not  initially  dominate  all  others,  the  advantages  and 
disadvantages  are  reconsidered  relative  to  the  other  candidates.  Mental  operations  restructure  the 
information  about  alternatives  until  one  candidate  seems  best  from  every  perspective— in  essence  a 
qualitative  sensitivity  analysis.  If  the  promising  candidate  then  dominates  the  other  alternatives,  it  is 
chosen.  Otherwise,  the  entire  process  is  repeated  by  selecting  another  promising  candidate  or  the  relevant 
information  is  reexamined  to  see  if  the  favored  alternative  can  become  dominant. 

This  description  and  its  variations  (e.g..  Beach,  1990)  is  consistent  with  how  people  appear  to 
make  some  choices,  particularly  when  heuristics  are  used  to  guide  the  process.  Moreover,  the  promising 
alternative  is  often  picked  out  early  in  the  process.  However,  personal  values  play  an  important  heuristic 
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role  in  structuring  the  process,  which  can  lead  to  selective  perceptions.  The  quality  of  the  process 
depends  on  the  early  phases  for  selection  of  the  attributes  that  are  relevant  to  the  problem.  Thus  the 
decision  maker's  values  about  what  is  important  influences  solution  effectiveness. 

Recognition 

Research  has  demonstrated  that  problem  solvers  rely  on  prior  experience  to  find  a  workable 
solution.  The  first  option  considered  is  often  an  acceptable  solution  which  can  then  be  evaluated  and 
developed  (Klein,  1989). 

The  effectiveness  of  the  solution  largely  depends  on  the  problem  solver's  level  of  experience  in 
the  domain.  Early  recognition  of  an  effective  candidate  is  a  process  that  has  also  been  associated  with 
expert  problem  solving.  This  approach  is  often  used  when  time  pressure  exists,  when  the  situation  is 
ambiguous  and  unstable.  Further,  this  approach  is  often  used  when  the  information  is  acquired 
perceptually  (that  is,  the  information  does  not  require  synthesis  and  analysis  much  beyond  initial 
perception),  when  information  is  continuous  rather  than  discrete,  and  when  information  is  simultaneously 
displayed  to  the  problem  solver  (that  is,  multiple  cues  are  experienced  at  the  same  time)  (Hammond, 
1993). 

Approaches  Compared 

In  sum,  a  variety  of  ways  have  been  proposed  to  describe  and  prescribe  human  problem  solving 
and  decision  making.  The  four  characterized  here  vary  primarily  in  their  reliance  on  different 
components  of  the  problem  solving  process  to  ensure  effective  solutions.  Procedural  models  emphasize 
the  importance  of  a  sequential  framework.  Analytic  models  rely  on  thorough  assessment  of  information. 
Dominance  models  hinge  on  structuring  the  process  using  one's  values.  Recognition  models  emphasize 
use  of  prior  experience. 

Both  analytic  and  procedural  approaches  are  prescriptive  in  that  they  emphasize  that  the  best 
solution  will  result  when  the  problem  solver  follows  a  particular  process  of  making  decisions  and  solving 
problems.  Both  these  methods  advocate  that  the  problem  solver  define  a  set  of  candidate  solutions  which 
are  then  compared.  When  a  procedural  or  analytic  method  is  used,  the  use  itself  is  often  taken  as  proof 
that  the  resulting  choice  is  the  optimal  alternative.  On  the  other  hand,  recognition  and  dominance 
approaches  propose  that  the  problem  solver  select  a  good  option  to  develop. 

Moreover,  the  success  of  all  four  approaches  depends  on  one's  prior  knowledge  about  the 
situation.  Although  Klein  (1993)  acknowledges  that  both  analytic  and  recognition-based  approaches  can 
be  useful,  it  is  not  clear  how  prior  knowledge,  one’s  approach  to  the  problem,  and  strategy  use  interact  to 
enable  the  solution  of  dynamic  and  non-routine  problems.  Both  recognition  and  dominance  are 
descriptive  models  of  how  some  people  have  solved  problems.  However,  they  provide  little  guidance 
about  how  to  do  it  better.  Moreover,  if  analytic  and  procedural  approaches  are  not  sufficient  in  uncertain, 
dynamic  situations  and  if  recognition  cannot  be  applied,  then  what  approach  is  employed  when  the  stakes 
are  high  and  there  is  not  time  for  detailed  analysis? 

A  more  robust  representation  takes  the  perspective  that  one  method  does  not  fit  all  problem 
situations.  Rather,  problem  solvers  approach  problems  in  different  ways  and  have  a  variety  of  candidate 
strategies  available  that  can  be  tailored  to  the  needs  of  the  specific  problem  (Payne  et  al.,  1988;  Pounds  & 
Fallesen,  1994).  In  this  view,  many  factors  would  influence  the  tailoring  process  such  as  personal 
experiences,  domain  knowledge,  type  of  problem,  problem  context,  and  personal  preferences. 
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Effectiveness  of  outcomes  can  also  be  influenced  by  the  problem  solver's  experience  in  the  problem 
domain,  task  factors  (such  as  amount  of  information  available),  and  situational  factors  (such  as  time 
available  and  consequences  of  the  outcome). 


Research  Questions 

From  an  overview  of  problem  solving  and  decision  making  literatures,  four  research  questions 
were  identified  (Pounds  &  Fallesen,  1994). 

1 .  How  do  military  leaders  approach  tactical  problems? 

2.  What  factors  influence  approach  use  (e.g.,  type  of  problem,  situational  factors)? 

3.  What  strategies  are  used? 

4.  Is  strategy  use  related  to  the  solution  that  is  selected? 

Question  #1:  How  Do  Military  Leaders  Approach  Tactical  Problems? 

Skilled  problem  solving  is  highly  valued  for  military  tactical  planning  and  conduct  of  battle 
(Fallesen,  1993;  Halpin,  1995;  Rumsey,  1995).  The  military  services  have  largely  relied  on  methods  that 
call  for  objective,  exhaustive,  and  systematic  comparison  of  options.  The  model  taught  during  Army 
officers’  instruction  in  tactical  planning  suggests  that  an  analytical  approach  is  best:  generate  multiple 
alternatives,  assess  each  independently,  and  then  compare  them,  selecting  the  “best”  course  of  action. 
However,  a  review  of  tactical  problem  solving  findings  discovered  that  commanders  did  not  closely 
conform  to  these  procedures  (Fallesen,  1993). 

Although  a  variety  of  techniques  to  promote  "rational"  choice  have  been  based  on  prescriptive 
utility  models,  these  techniques  are  often  too  rigid  and  time  consuming  to  be  appropriate  for  finding 
solutions  in  complex,  changing  environments.  Simplistic  procedural  models  or  exhaustive  comparisons 
are  not  appropriate  or  sufficient  for  the  complexities  of  actual  situations  (Fallesen,  1993).  Prescriptive 
decision  analytic  models  were  found  to  be  limited,  particularly  when  planning  in  situations  of  uncertainty 
(Fallesen,  1993;  Fallesen,  Carter,  Perkins,  Michel,  Flanagan,  &  McKeown,  1992). 

In  a  comprehensive  review  of  research  on  the  Army  planning  process,  Fallesen  (1993)  found 
evidence  that  actual  problem  solving  was  disconnected  from  prescribed  processes.  Effective  solutions 
depended  on  the  quality  rather  than  frequency  of  procedures.  Goals,  constraints,  and  situation  knowledge 
rather  than  systematic  methods  governed  option  selection.  These  findings  suggest  that  instruction  should 
include  these  factors.  Just  what  would  constitute  this  alternate  instruction  has  yet  to  be  identified. 
However,  a  promising  course  of  Practical  Thinking  instruction  has  been  developed  and  tested  (Fallesen, 
Michel,  Lussier,  &  Pounds,  1996). 

By  examining  how  military  leaders  actually  approach  tactical  problems  and  by  identifying  the 
strategies  that  they  use  to  reach  a  workable  solution,  the  question  of  what  a  more  relevant  training  method 
might  look  like  can  begin  to  be  answered.  From  this,  strengths  and  likely  error  points  in  the  strategies 
used  can  be  identified  and  appropriate  instructional  support  designed. 

Question  #2:  What  Factors  Influence  Approach  Use? 

Individual,  situational,  and  task  characteristics  have  been  shown  to  influence  how  people 
approach  problems.  Although  people  generally  demonstrate  limited  capacities  for  processing  information 
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about  their  environment,  research  has  demonstrated  that  people  can  adapt  to  task  demands  and 
characteristics  of  decision  situations. 

When  situations  have  multiple,  conflicting  goals,  people  have  demonstrated  the  flexibility  to 
make  trade-offs  between  accuracy  and  effort  (Payne,  Bettman,  &  Johnson,  1993).  In  situations  where 
time  pressure  exists,  people  have  demonstrated  that  they  spend  less  time  thinking  about  the  problem.  One 
can  speculate  that  this  is  due  to  “thinking  faster”  while  still  considering  ail  the  options,  evaluating  them 
on  all  important  attributes,  and  selecting  the  one  with  the  highest  resulting  value.  This  argument  assumes 
that  decision  makers  are  still  using  analytic  methods  (Orasanu  &  Connolly,  1993).  However,  rather  than 
“thinking  faster,”  skilled  problem  solvers  appear  to  focus  on  the  type  of  problem,  applying  their  prior 
knowledge  about  similar  situations  to  the  problem  at  hand  by  recognizing  the  situation  as  a  previously 
solved  problem  or  by  developing  a  promising  alternative,  by  selecting  only  the  most  relevant  information 
to  think  about,  considering  only  a  few  options,  or  changing  strategies.  Moreover,  performance  in  time 
pressured  decisions  has  been  demonstrated  to  improve  with  experience  in  the  problem  domain  (Klein, 
Calderwood,  &  Clinton-Cirrocco,  1986;  Payne  et  al.,  1993). 

In  addition  to  accuracy,  effort,  and  prior  knowledge,  the  perceived  severity  of  the  consequences 
of  the  outcome  also  influences  problem  solving.  Higher  stakes  increase  indecision  and  the  tendency  to 
persist  with  the  first  decision  once  it  is  made  (Lipshitz,  1993).  Tactical  problems  generally  occur  in  these 
types  of  time  pressured,  high  stakes  situations. 

Question  #3:  What  Strategies  Are  Used? 

Judgment  and  decision  making  literatures  abound  with  proposed  strategies  to  solve  problems  (see 
Pounds  &  Fallesen,  1994).  Because  strategies  for  organizing  information  are  an  inherent  part  of  our 
thinking,  they  have  been  studied  in  a  variety  of  domains.  However,  strategies  can  facilitate  or  hinder  the 
problem  solving  process.  Facilitation  occurs  by  knowing  how,  when,  and  where  to  apply  one’s 
knowledge  using  an  appropriate  strategy.  Problem  solving  is  hindered  when  knowledge  is  misapplied  or 
when  an  inappropriate  strategy  is  applied. 

Payne  et  al.  (1993)  reviewed  studies  which  found  that  strategy  selection  was  partially  contingent 
on  attributes  of  the  problem.  In  the  presence  of  a  large  set  of  information,  people  might  selectively  attend 
to  only  the  relevant  information;  however,  it  has  also  been  demonstrated  that  irrelevant  information  can 
influence  the  process  (Gaeth  &  Shanteau,  1984).  The  amount  of  information  can  also  influence  the  use  of 
trade-offs.  As  task  complexity  increases,  people  tend  to  rely  on  simpler  strategies. 

Although  strategy  use  has  been  associated  with  approach,  little  work  has  been  done  to  identify 
which  strategies  are  unique  to  a  particular  approach.  For  example,  identification  of  relevant  aspects  is 
associated  with  the  use  of  a  recognition  approach.  However,  relevant  aspects  may  also  be  part  of  the 
successful  use  of  the  dominance  approach.  Another  example  would  be  the  strategy  of  relying  on 
quantitative  assessments.  Both  analytic  and  dominance  approaches  incorporate  this  strategy,  but  in 
varying  degrees.  Thus,  rather  than  one  set  of  strategies  comprising  one  approach,  it  is  more  likely  the 
case  that  the  same  strategy  is  incorporated  into  different  approaches. 

Question  #4:  Is  Strategy  Use  Related  to  the  Solution  That  is  Selected? 

In  other  words,  does  it  matter  which  strategy  is  used?  A  “process”  perspective  proposes  that  the 
strategies  which  the  problem  solver  employs  to  reach  a  solution  should  be  the  main  focus  (Ford,  Schmitt, 
Schechtman,  Hults,  &  Doherty,  1989).  This  perspective  emphasizes  the  importance  of  information 
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acquisition  and  analysis,  and  the  importance  of  minimizing  error  points  in  the  process  to  arrive  at  a  better 
decision.  An  “outcome”  perspective,  on  the  other  hand,  focuses  on  the  decision  irrespective  of  how  one 
gets  there. 

Process  and  outcome  perspectives  need  not  be  mutually  exclusive,  however.  Focus  on  how 
differences  in  process  relate  to  differences  in  outcomes  would  illustrate  how  one’s  strategy  influences 
one’s  decision.  Different  strategies  may  be  associated  with  different  solutions. 

Research  using  skilled  problem  solvers  has  demonstrated  that  they  use  different  strategies  to 
reach  solutions  than  do  the  less  skilled  (Klein  et  al.,  1986).  Further,  skilled  problem  solvers  make  better 
quality  decisions,  often  by  tolerating  small  errors  in  order  to  avoid  making  large  errors.  Specific 
differences  in  strategy  usage  between  more  and  less  skilled  problem  solvers  could  be  explicitly  included 
in  training  programs  as  domain  knowledge  increases,  rather  than  just  hoping  that  the  more  effective 
strategies  will  emerge  spontaneously. 

Purpose 


The  purpose  of  this  research  was  to  examine  how  military  leaders  go  about  solving  tactical 
problems  and  to  identify  the  strategies  they  used  to  reach  a  solution.  This  was  accomplished  by 
observing  how  military  officers  proceeded  to  find  solutions  to  sample  tactical  problems.  Participants 
were  guided  through  interviews  during  which  they  described  their  thoughts  while  solving  problems. 
They  also  identified  thinking  strategies  that  were  useful  to  them. 

The  goal  is  to  use  information  about  the  approaches  and  strategies  of  more  experienced  problem 
solvers  to  leverage  training  and  instruction  for  the  less  skilled.  In  this  way,  potential  error  points  can  be 
countered. 


Method 


Participants 

To  address  these  questions,  data  were  collected  from  82  U.S.  Army  officers  ranging  in  rank  from 
2nd  Lieutenant  («  =  5)  and  Lieutenant  (n  =  2)  to  Lieutenant  Colonel  (n  =  3).  Most  officers  were 
Captains  (n  =  5 1  )  or  Majors  («  =  21).  Time  in  service  ranged  from  1  to  19  years.  Eighty  officers 
participated  in  the  interviews.  Two  participants  contributed  only  partial  data. 

Materials 

Four  paragraph- length  descriptions  were  written  to  represent  the  four  types  of  approaches  that 
one  might  adopt  to  solve  a  problem:  the  procedural  approach  (ST  101-5, 1994),  the  decision  analytic 
approach  (Anderson  et  al.,  1981),  the  recognition-based  approach  (Klein,  1989),  and  the  dominance 
approach  (Montgomery,  1989).  The  descriptions  are  listed  in  the  Appendix  A. 

Two  problems  were  constructed.  One  problem  placed  the  participant  in  the  role  of  an 
mechanized  infantry  battalion  commander  whose  mission  was  to  cross  a  bridge  held  by  friendly  forces 
and  establish  a  bridgehead  in  preparation  for  an  offensive  the  following  day.  However,  the  "commander" 
receives  new  information  and  must  decide  on  whether  modifications  are  needed  in  the  course  of  action. 
Most  participants  were  expected  to  be  familiar  with  this  type  of  problem.  The  second  problem  placed  the 
participant  in  the  role  of  an  infantry  company  commander.  His  company  has  been  training  in  another 
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country  when  he  is  called  into  Battalion  headquarters  and  assigned  to  rescue  a  U.S.  ambassador  and  his 
family  from  their  captors.  Few  participants  were  expected  to  be  familiar  with  this  type  of  problem. 

Problems  that  were  likely  to  be  more  and  less  familiar  were  used  to  allow  participants  to  exercise 
their  problem  solving  skills.  Both  problems  included  a  deadline  required  by  the  mission.  Problems  were 
also  characterized  by  prototypical  tactical  elements,  such  as  mission,  use  of  force,  and  presence  of  enemy 
elements.  The  problems  differed  in  focus.  The  bridge  problem  was  more  typical  for  an  infantry 
battalion,  while  the  hostage  situation  was  more  unusual  for  a  unit  other  than  Ranger  or  Special  Forces. 
The  bridge  problem  was  terrain-oriented  while  the  hostage  problem  was  rescue-oriented.  It  was  expected 
that  military  leaders  would  have  different  levels  of  experience  and  knowledge  about  these  two  types  of 
problems. 

Sixty-six  strategies  were  identified  from  various  literatures.  See  Pounds  and  Fallesen  (1994)  for 
an  elaborated  listing.  A  subset  of  48  were  selected  based  on  the  criterion  of  minimizing  redundant 
aspects  of  definitions.  These  were  29  information  processing  strategies  and  19  choice  strategies.  A  brief 
everyday  definition  of  each  strategy  was  put  on  individual  4x6  index  cards  so  the  participants’  attention 
and  discussion  could  be  focused  on  individual  strategies.  For  example,  a  decomposition  strategy  was 
represented  by  Broke  the  problem  into  smaller  problems.  The  complete  set  of  descriptions  is  listed  in 
Appendix  B. 

Procedure 

Participants  were  each  scheduled  for  one  three-hour  interview,  with  approximately  equal  time 
allotted  to  discuss  each  of  three  problems.  Each  interview  was  recorded.  First,  the  rationale  and 
background  for  the  study  was  discussed  with  each  participant.  Then  each  completed  a  demographic 
questionnaire. 

Participants  first  discussed  a  problem  that  they  had  personally  faced  and  solved.  This  served  to 
“loosen  up”  the  participants  and  to  familiarize  them  with  the  procedure.  Participants  were  assured  that 
for  each  problem  the  researcher's  primary  interest  was  not  in  the  correctness  of  the  decision  but  in  the 
thinking  process  that  resulted  in  their  solution. 

Following  each  discussion,  participants  completed  a  card  sorting  task.  Sixteen  officers  sorted 
into  three  categories  (yes,  no,  uncertain).  These  were  rescaled  and  included  with  the  rest  of  the  data. 
Sixty-four  participants  sorted  the  strategy  cards  into  six  categories;  not  used,  uncertain,  used  (but  not 
important),  important,  very  important,  and  most  important.  Sorting  was  based  on  the  participant’s 
subjective  judgment  of  how  important  the  strategy  was  in  solving  the  problem.  Groupings  were  recorded. 

Participants  were  then  presented  with  the  first  of  the  two  tactical  problems  and  asked  to  talk  aloud 
as  they  proceeded  to  “tell  us  what  you  would  do”  in  the  scenario.  Order  of  problems  was 
counterbalanced  across  participants.  When  the  participant  reached  a  solution,  the  card  sorting  task  was 
again  performed.  The  second  tactical  problem  was  then  presented  and  the  process  repeated. 

After  completing  the  three  problems  and  card  sorting  tasks,  participants  read  the  four  descriptions 
of  problem  solving  approaches  and  assigned  100%  across  the  four  descriptions  to  represent  how  each 
approach  figured  in  their  problem  solving  process.  The  only  requirement  was  that  the  total  percentage 
across  all  four  descriptions  summed  to  1 00.  This  method  allowed  participants  to  indicate  relative  usage 
of  approaches  within  the  problem  solving  process  without  restricting  their  selection  to  one  over  the 
others.  Any  concerns  or  questions  about  the  meaning  of  the  descriptions  were  addressed  by  the 
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interviewer  prior  to  final  point  assignment.  If  the  participant  thought  that  a  fifth  description  was  needed 
to  describe  his  process,  he  assigned  points  to  a  fifth  option.  Thirty-two  participants  did  this  assignment 
only  once  for  problem  solving  in  general  after  they  finished  all  three  problems.  Forty-eight  participants 
did  this  rating  before  discussing  the  problems.  Forty-six  participants  also  made  these  assignments  after 
each  problem.  All  participants  then  listed  any  factors  they  thought  influenced  their  use  of  one  approach 
over  another. 

To  identify  other  factors  that  might  affect  the  tactical  problem  solving  process,  the  first  sample  of 
participants  («  =  32)  was  asked  to  identify  other  possible  elements.  The  108  responses  reflected 
participants'  concerns  about  the  amount  of  time  available  to  find  a  solution  and  plan  a  course  of  action, 
the  importance  of  the  mission,  the  type  of  problem  and  their  familiarity  with  it,  and  the  resources  and 
constraints  that  were  present.  Based  on  these  comments,  two  additions  to  the  materials  and  procedures 
were  incorporated  for  the  last  48  participants.  After  solving  each  tactical  problem,  participants  made  a 
subjective  judgment  and  distributed  the  100%  percentage  points  across  the  descriptions  of  the  four 
approaches  to  show  the  percentage  that  each  approach  was  used  in  the  solution  process  for  that  problem. 
Participants  also  gave  a  subjective  ratings  on  Likert  scales  to  each  of  the  following  four  questions  asked 
after  each  problem.  For  each  question,  the  scale  was: 

0...1...2...3...4...5...6...7...8...9...10 
Very  low  Very  high 


(1)  How  familiar  are  you  with  this  type  of  problem/situation?  In  other  words,  how  much 
experience  or  knowledge  do  you  have  with  this  type  of  situation?  Both  experience  and 
knowledge  were  incorporated  into  this  question  although  some  have  suggested  that  these 
components  are  separable  (Alba  &  Hutchison,  1987). 

(2)  How  important  (or  critical)  is  the  outcome  of  the  situation?  In  other  words,  how  high  are  the 
stakes?  This  question  was  intended  to  capture  the  importance  that  participants'  attached  to  the 
consequences  of  their  proposed  actions. 

(3)  How  much  time  did  you  feel  was  available  before  you  had  to  commit  to  a  decision?  Both 
problems  incorporated  a  time  deadline  for  planning.  Thus  this  question  was  intended  to  capture 
participants'  perceptions  of  time  pressure. 

(4)  How  much  mental  effort  for  thinking  did  it  take  to  identify/resolve  this  problem?  This 
question  required  participants  to  estimate  how  much  thinking  they  devoted  to  the  process. 

Results  and  Discussion 

Eighty-two  officers  participated  in  the  data  collection.  However,  some  had  incomplete  data 
because  of  unforeseen  events,  such  as  scheduling  changes  and  unexpected  duty  requirements,  which 
shortened  their  interviews.  Also,  some  materials  were  incorporated  into  the  interviews  for  only  some 
subgroups  as  the  project  evolved.  For  example,  the  first  32  officers  assigned  points  to  their  overall 
problem  solving  approach.  The  next  48  were  also  asked  to  give  ratings  for  their  approach  to  each 
problem  in  addition  to  their  general  approach.  Thus,  some  of  the  analyses  were  conducted  for  these 
subgroups. 


8 


How  Military  Leaders  Approach  Problems 

When  asked  how  they  generally  preferred  to  approach  problems,  the  82  participants’  distributed 
responses  over  all  four  approaches.  These  ratings  were  not  made  for  any  particular  problem  but  for  their 
generally  preferred  approach  to  solving  whatever  problems  they  face.  Some  participants  made  comments 
like:  “In  a  perfect  world,  I  would....” 

Review  of  the  percentages  assigned  to  the  four  descriptions  showed  that  most  participants  used 
more  than  one  approach  to  solve  problems.  Some  participants  responded  that  they  preferred  not  to  use 
the  analytic,  procedural,  recognition  or  dominance  approaches  in  their  problem  solving  processes  at  all. 
The  range  of  results  for  each  approach  was: 


Analytic 

0-  80% 

Procedural 

0-100% 

Recognition 

0-  80% 

Dominance 

0-  60% 

Examination  of  95%  confidence  intervals  around  the  mean  number  of  points  assigned  to  each 
approach  (Gibbons,  1993)  showed  that  participants  reported  using  analytic  and  dominance  approaches 
significantly  less  than  either  the  procedural  or  recognition  approach  (Figure  1). 

Three  participants  did  not  distribute  the  entire  100  points  across  the  four  presented  approaches, 
preferring  instead  to  also  give  points  to  a  hypothetical,  unnamed  fifth  approach  that  they  posited  might 
exist.  For  the  general  approach  ratings,  one  participant  gave  the  fifth  category  one  point.  For  the 
Hostage  problem,  two  participants  rated  a  fifth  category  when  responding  to  their  approach.  Those 
problem-specific  responses  were  not  included  in  the  approaches  to  general  problem  solving  shown 
(Figure  1  and  Table  1). 

Examples  illustrate  how  participants  elaborated  the  four  approaches.  One  participant  gave  the 
analytic  approach  30  points,  procedural  40  points,  recognition  20  points,  and  dominance  10  points  when 
describing  his  general  problem  solving  approach.  However  he  noted  two  changes  to  the  procedural 
approach’s  description. 

“Use  screening  and  use  the  next  best  course  of  action  instead  of  starting  over.  If  time  constrained 
then,  reverse  the  order  of  these  (he  would  reverse  his  ordering  of  point  assignments).” 

The  one  participant  who  assigned  all  100%  to  a  fifth  category  for  his  approach  to  the  Hostage 
problem  took  parts  of  the  other  descriptions  and  added  to  them  to  make  his  own  as  follows.  From  the 
analytic  approach  he  used  nothing.  He  indicated  that  reconsideration  (from  the  dominance  approach 
description)  and  recognizing  familiar  situations  (from  the  recognition  approach  description)  were 
somewhat  important.  However,  his  main  points  were: 

“Clarify  the  situation  and  gather  information  without  making  Judgments  about  it  (from  the 
procedural  approach  description).  Consequences  of  using  the  usual  solution  are  imagined  to 
identify  pitfalls  (from  the  recognition  approach  description).  Develop  [a]  best  sub-optimal 
solution  and  work  out  details.” 
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However, 

correlations  between  the 
percentages  of  reported 
use  assigned  to  each 
approach  showed  that  use 
of  a  recognition-based 
approach  was  negatively 
related  to  use  of  an 
analytic  or  a  procedural 
method.  Use  of  a 
dominance  approach  was 
also  negatively  related  to 
use  of  both  analytic  and 
procedural  approaches. 
Correlations  (and 
probability  values)  are 
shown  in  Table  1 .  These 
relationships  suggest  that 
when  people  employ  a 

detailed  utility  method  or  adopt  a  stepwise  procedure,  the  two  methods  that  emphasize  development  of  a 
likely  candidate  were  less  likely  to  be  rated  as  used.  Approaches  that  give  the  problem  solver  a  method  to 
rely  on  may  be  employed  when  a  likely  solution  is  not  immediately  available.  However,  relying  on 
procedure  alone  does  not  necessarily  guarantee  a  good  solution  (e.g.,  if  the  quality  of  information  used  in 
the  procedure  is  flawed  or  contradictions  are  not  recognized). 

Table  1 

Correlations  Between  Approaches  Based  on  Percent  of  Preferred  Use  {n  -  82) 


40 

35 


I  30 


25 


1 

I 

I  20 

ll5 

I  10 

5 

0 


Aiaiytic 


Recogniticn 


Approach 


Figure  1 

95%  Confidence  Interval  Around  the  Mean  for  Each  Approach  (n  =  82) 


Analytic 

Procedural 

Recognition 

Dominance 

Time  in  Service 

.10  (.38) 

-.10  (..37) 

.05  (.65) 

-.03  (.80) 

Analytic 

-.19  (.09) 

-.40  (.0002) 

-.32  (.004) 

Procedural 

-.55  (.0001) 

-.49  (.0001) 

Recognition 

-.02  (.88) 

U.S.  Army  doctrine  and  training  advocates  procedural/analytic  approaches  for  problem  solving 
and  decision  making.  Thus,  one  could  expect  that  participants’  time  in  service  (TIS)  would  be  positively 
related  to  use  of  those  approaches.  Nevertheless,  reported  use  of  the  four  approaches  was  not  correlated 
with  participants'  length  of  service.  The  relationships  in  Figure  1  support  the  idea  that  military  leaders 
prefer  to  use  a  combination  of  approaches  to  find  solutions.  However,  it  is  not  clear  when  they  call  on 
different  methods  and  what  other  approaches  can  be  described  beyond  the  four  used  for  the  present  study. 

Preliminary  analyses  to  clarify  how  another  approach  might  be  described  were  conducted  by 
Fallesen  (1996).  A  cluster  analysis  performed  on  the  strategy  importance  ratings  from  a  subset  of  32 
participants  suggested  clusters  of  strategies  based  on  different  types  of  thinking  processes:  framing  the 
problem,  information  quality,  looking  at  the  “big  picture,”  comparison,  objective  evaluation,  subjective 
evaluation,  and  reconsideration. 
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Influences  on  Approach  Use 

Because  military  leaders  are  more  routinely  exposed  to  problems  of  maneuvering  forces  while 
smaller  numbers  typically  train  for  hostage  rescue  scenarios,  it  was  hypothesized  that  the  Bridge  problem 
would  be  more  familiar  than  the  Hostage  problem.  Thus,  participants  would  be  more  likely  to  adopt  a 
recognition  approach  to  solve  the  Bridge  problem  than  the  Hostage  problem. 

Other  variables  were  also  hypothesized  to  influence  how  participants  approached  the  scenarios. 
Increased  effort  devoted  to  the  problem  solving  task  would  be  positively  related  to  use  of  an  analytic 
approach.  Time  available  would  be  negatively  related  to  use  of  a  recognition  approach.  Payne  et  al. 
(1993)  proposed  that  the  importance  of  the  decision  leads  to  greater  effort  expended  in  making  the 
decision.  Thus,  the  importance  of  the  outcome  would  be  positively  related  to  effort.  However,  the 
“decision  about  how  to  decide”— similar  to  what  we  are  calling  an  “approach”—  is  also  contingent  on  a 
host  of  other  task  factors  related  to  effort-accuracy  trade-offs. 

Forty-eight  officers  responded  to  the  scaled  questions  for  familiarity,  effort,  time,  and  stakes  for 
both  the  Bridge  and  Hostage  problem.  Excluded  were  the  post-training  ratings  for  one  of  the  tactical 
problems  by  officers  who  participated  in  a  training  session. 


As  a  manipulation 
check,  familiarity  ratings  for  the 
Bridge  and  Hostage  problems 
were  compared  using  analysis  of 
variance.  The  Bridge  scenario 
was  rated  significantly  more 
familiar  (Af=6.75)  than  the 
Hostage  scenario  (Af=4.42),  Ffl, 
61)=14.12,  M5'e=6.06,/><.0004. 
Differences  in  responses  to  the 
problems  are  illustrated  in 
Figure  2  as  cumulative 
percentages. 


Analyses  of  variance 
showed  that  the  type  of  problem 
made  a  significant  difference  in 
participants'  reported  use  of  both 
a  recognition  approach,  Ffl, 
61)=20.7,A^e=816.32, 

/K.OOOl  and  the  dominance 
approach,  F(\,  61)=  5.91,  MSe=  814.05,/7<.018.  This  supports  the  proposal  that  one’s  use  of  a 
dominance  approach  is  also  linked  to  using  one's  experiences  (or  lack  of  experience)  in  the  problem 
domain.  Problem  type  was  not  a  significant  effect  on  the  reported  use  of  either  the  analytic  or  the 
procedural  approaches.  Point  distributions  for  each  approach  by  problem  are  shown  in  Figure  3.  Thus, 
the  familiarity  of  the  officer  with  the  problem  appears  to  be  one  factor  that  influences  the  approach 
which  he  or  she  adopts  to  find  a  solution. 

Participants'  ratings  of  familiarity  with  the  problem  were  positively  related  to  the  use  of  a 
recognition  approach  (r  =.50,/><.0001).  Moreover,  familiarity  was  negatively  related  to  a  dominance 


Figure  2 

Distribution  of  Familiarity  Ratings  by  Problem 
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approach  (r  =-.45,/K.0002).  Familiarity  with  the  problem  was  not  significantly  related  to  either  an 
analytic  or  procedural  approach.  Instead  of  using  a  procedural  approach  for  unfamiliar  problems  some 
participants  used  dominance,  perhaps  in  the  form  of  using  a  questioning  strategy  or  sensitivity  analysis 
(e.g.,  “What  would  happen  if  this  option  were  dominant?”  or  “What  would  happen  If  this  aspect  was  most 
important?”).  Another  possibility  is  that  in  the  absence  of  domain  specific  experience  to  apply  to  the 
problem,  the  problem  solver  tried  to  transfer  and  apply  knowledge  from  other  domains. 

This  finding 

demonstrates  that  familiarity 
with  the  problem  domain 
influences  how  one  chooses  to 
approach  the  problem. 
Although  the  relationship 
between  familiarity, 
experience,  and  use  of  a 
recognition  approach  has  also 
been  demonstrated  in  other 
research  (e.g.,  Klein  et  al., 
1993)  the  relationship 
between  familiarity, 
experience,  and  dominance  is 
less  clear.  Something  akin  to 
familiarity  or  experience  must 
be  at  work  during  the 
structuring  processes  of  the 
dominance  approach  to  enable 

Based  on  findings  from  studies  of  schema-based  reasoning  and  effort-accuracy  trade-offs  (Payne 
et  al.,  1993),  it  was  hypothesized  that  use  of  an  analytic  approach  would  be  positively  related  to 
participants'  subjective  effort  ratings.  This  was  the  case.  Use  of  the  analytic  approach  was  positively 
correlated  with  effort  ratings  (r  =.28,/?<.03).  Moreover,  effort  ratings  were  negatively  related  to  use  of  a 
recognition  approach  (r  =  -.30,/?<.02). 


the  dominance  process  to  develop  a  solution  at  all. 
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(4)  Examination  of 
responses  to  the  time 
available  question  showed 
little  variability  across  the  two 
tactical  scenarios.  Most 
responses  were  toward  the 
Very  Low  end  of  the  scale  for 
both  problems  (Figure  4). 
Participants’  ratings  of  time 
available  were  not 
signiflcantly  related  to  any  of 
the  four  approaches. 
Differences  in  responses  to 
the  question  of  available  time 
are  illustrated  in  Figure  4  as 
cumulative  percentages. 


(5)  Examination  of  the 

responses  to  the  importance 
question  showed  a  lack  of 
variability  across  the  two 
tactical  problems.  Most 
participants  responded  with 
scale  values  toward  the  Very 
High  end  of  the  scale  for  both 
the  Bridge  and  Hostage 
problems  (Figure  5).  The 
perceived  criticality  of  the 
outcome  was  not 
significantly  related  to  use  of 
any  of  the  approaches. 

In  sum,  based  on  the 
analyses  of  the  ratings  of  the 
approaches,  it  appears  that 
whichever  approach  is 

adopted  by  the  problem  solver  to  find  a  solution  depends  on  several  factors.  These  results  are  in  line  with 
the  findings  of  Fallesen  (1993)  by  demonstrating  that  officers  do  not  unilaterally  adopt  the  “schoolhouse” 
procedural  method  when  solving  problems.  The  notion  that  people  use  a  combination  of  approaches  to 
reach  a  solution  may  be  closer  to  reality.  Moreover,  other  approaches  might  await  description.  If  so, 
then  future  investigations  should  aim  to  identify  other  approaches,  define  them,  and  show  how  they  work. 

The  type  of  problem  that  the  officer  confronts  appears  to  be  one  factor  that  influences  the 
approach  which  he  or  she  adopts  to  find  a  solution.  To  explore  that  idea,  the  two  scenarios  used  for  the 
interviews  were  selected  to  represent  a  range  of  more  and  less  familiar  problem  domains  and  constraints. 
However,  it  would  be  impossible  to  foresee  every  problem  that  military  leaders  might  encounter  on  the 
battlefield,  in  garrison,  and  particularly  during  operations  other  than  war.  Thus,  training  would  be 
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leveraged  by  an  understanding  of  how  military  officers  marshal  their  existing  knowledge-whatever  its 
level-to  handle  the  uncertainty  in  situations  where  a  rule  cannot  be  applied.  For  example,  rules  or 
principles  might  be  modified,  or  analogies  made  with  remotely  associated  instances.  Partial  procedures 
from  other  instances  might  be  combined. 

The  level  of  one’s  prior  knowledge  appears  to  play  an  important  role  in  both  recognition  and 
dominance  approaches.  However,  whether  using  the  dominance  approach  compensates  for  lack  of 
knowledge  is  not  certain.  Dominance  processes  may  resemble  hypothesis  testing  (e.g.,  “Is  this  course  of 
action  the  one?”).  Possibly,  another  approach  shares  some  of  the  features  of  both  dominance  and 
recognition  and  can  be  described  as  the  process  of  working  with  the  less  familiar  to  make  it  more 
familiar.  If  so,  understanding  how  this  process  works  could  be  used  to  enhance  how  one  applies  one’s 
knowledge  to  problems. 

Strategies  Used 

Tactical  problem  solving  was  also  examined  at  the  strategy  level.  In  addition  to  enhancing  the 
problem  solver’s  overall  approaches  to  problems,  tactical  problem  solving  can  also  be  supported  by 
examining  which  particular  strategies  are  crafted  together  when  solving  a  problem.  Strategies  have 
strengths  which  can  be  developed  where  they  are  lacking  and  weaknesses  which  can  be  addressed  when 
they  are  present.  Each  strategy  which  is  used  would  provide  an  opportunity  for  training  and  support.  For 
example,  those  who  are  skilled  in  solving  problems  in  their  domain  are  better  able  to  separate  the 
important  information  from  the  irrelevant.  Nevertheless,  even  skilled  problem  solvers’  methods  have 
been  enhanced  through  training  by  pointing  out  how  they  were  using  meaningless  information  (Gaeth  & 
Shanteau,  1984).  Another  example  of  how  strategy  use  influences  the  problem  solving  process  is  the 
strategy  of  breaking  the  problem  into  smaller  problems.  This  strategy  works  well  to  structure  difficult  or 
complex  problems  (Newell  &  Simon,  1972).  However,  this  strategy  is  time  consuming  when  the  problem 
constraints  or  relevant  aspects  of  the  problem  are  not  clear  and  it  consumes  most  of  the  problem  solving 
efforts  leaving  little  for  the  actual  solution  process  (Reitman,  1965;  Voss  &  Post,  1988). 

To  determine  the  general  types  of  strategies  being  used,  data  from  the  card  sorting  tasks 
following  the  tactical  problems  were  used.  Sorting  for  the  individuals’  own  problem  were  excluded 
because  recall  of  events  was  relatively  more  likely  to  be  influenced  by  long-term  memory  retrieval 
processes,  whereas  the  sorting  for  each  tactical  problem  was  done  immediately  after  it  was  discussed. 
Moreover,  the  types  of  problems  elicited  from  participants’  experience  varied  widely.  The  individuals’ 
recalled  problems  will  be  examined  separately  in  future  analyses.  Card  sorts  were  excluded  for  one 
problem  from  each  of  the  32  officers  who  participated  in  a  training  session  related  to  one  of  the  strategies. 

To  identify  strategies  that  were  generally  used,  means  were  calculated  for  all  of  the  strategy 
phrases  over  both  of  the  tactical  problems.  Forty-eight  participants  sorted  cards  for  both  tactical 
problems.  Pretraining  problems  for  another  32  participants  were  included.  Strategies  that  participants 
rated  as  3  (“Important”)  or  higher  are  listed  in  Table  2. 
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Table  2 

Strategy  Importance  Ratings  Over  Both  Tactical  Problems 


P18 

Identified  a  sjpedfic  goal  — •  

gi«ai 

4.33 

P21 

Identified  facts 

3.79 

1.32 

C15 

The  best  option  had  to  meet  certain  required  criteria 

3.58 

1.83 

P9  ^ 

Restated  the  problem  by  visualizing  or  drawing 

3.48 

1.45 

P19 

Identified  problems  in  accomplishing  the  goals 

3.37 

1.38 

P6 

Considered  what  information  was  missing  and  its  implications 

3.24; 

P22 

Identified  assumptions 

3.21 

1.44 

P12 

Looked  at  the  big  picture  before  the  details 

3.11 

1.72 

Pll 

Note.  N  = 

Broke  the  problem  into  smaller  problems 

128. 

3.04 

1.67 

To  identify  differences  in  strategy  use  by  problem,  mean  importance  ratings  were  calculated  for 
each  strategy  within  each  problem.  Strategies  that  participants  rated  as  3  or  higher  for  each  problem  are 
listed  in  Tables  3a  and  3b.  Hereafter,  ‘P’  indicates  a  processing  strategy.  ‘C’  indicates  a  choice  strategy. 
Two  strategies  were  important  in  the  Bridge  problem  (PI 2  and  P27)  but  not  in  the  Hostage  problem. 
Three  strategies  (PI  1,  P5,  Cl 6)  were  important  in  the  Hostage  problem  but  not  in  the  Bridge  problem. 

Table  3a 

Strategies  Rated  Greater  Than  or  Equal  to  3.00  for  the  Bridge  Problem 


P18 

Identified  a  specific  goal 

4.31 

1.43 

P21 

Identified  facts 

3.75 

1.29 

P9 

Restated  the  problem  by  visualizing  or  drawing 

3.52 

1.47 

P12 

Looked  at  the  big  picture  before  the  details  * 

3.38 

1.69 

C15 

The  best  option  had  to  meet  certain  required  criteria 

3.35 

1.94 

P6 

Considered  what  information  was  missing  and  its  implications 

3.30 

1.45 

P19 

Identified  problems  in  accomplishing  the  goals 

3.20 

1.44 

P22 

Identified  assumptions 

3.11 

1.56 

P27  Considered  the  accuracy  of  the  information  * 

Note.  *  =  non-overlapping  between  problems.  N  =  64. 

3.00 

1.52 
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Table  3b 

Strategies  Rated  Greater  Than  or  Equal  to  3.00  for  the  Hostage  Problem 


P18 

Identified  a  specific  goal 

4.3S 

1.26 

P21 

Identified  facts 

3.83 

1.36 

CIS 

The  best  option  had  to  meet  certain  required  criteria 

3.81 

1.69 

P19 

Identified  problems  in  accomplishing  the  goals 

3.S4 

1.31 

P9 

Restated  the  problem  by  visualizing  or  drawing 

3.44 

1.44 

P22 

Identified  assumptions 

3.32 

1.31 

Pll 

Broke  the  problem  into  smaller  problems  * 

3.23 

1.60 

P6 

Considered  what  information  was  missing  and  its  implications 

3.19 

PS 

Determined  parts  of  the  plan  that  would  be  prone  to  flaws* 

3.09 

1.S7 

C16 

The  best  option  should  have  met  certain  desired  criteria  * 

3.0S 

1.90 

Note.  ♦  =  non-overlapping  between  problems.  N  =  64. 


Participants  generally  agreed  (Table  2)  that  the  two  most  important  strategies  for  solving  both 
problems  were  Identified  a  specific  goal  (P 1 8)  and  Identified facts  (P2 1 ).  However,  this  does  not  mean 
that  the  goals  and  the  facts  were  the  same,  only  that  the  same  strategies  were  used.  The  ratings  did  not 
reveal  whether  participants’  goals  were  the  same  within  problems  or  whether  participants  identified  the 
same  facts  as  relevant.  The  ratings  suggest  only  that  these  strategic  processes  were  similarly  important. 
Only  three  strategies  (PIO,  P26,  and  PI 2)  differed  by  more  than  .5  rating  points  between  problems. 
Comparisons  of  each  strategy  between  problems  showed  no  statistically  significant  difference  in  strategy 
ratings  between  problems. 

A  straightforward  picture  of  strategy  use  would  allow  one  to  associate  a  unique  set  of  strategies 
with  each  approach.  However,  if  strategies  are  combined  constructively,  then  it  is  reasonable  to  assume 
that  some  strategies  are  operating  similarly  in  different  approaches.  It  might  not  matter  which  strategies 
are  used  so  long  as  a  good  solution  is  reached.  The  opposite  might  also  be  true:  using  strategies 
differently  results  in  different  solutions. 

Strategy  Use  Related  to  Solution 

To  clarify  whether  differences  in  strategy  use  mattered,  the  data  were  examined  to  determine 
whether  differential  use  of  strategies  would  result  in  different  solutions.  Only  solutions  for  the  Bridge 
have  been  analyzed.  Thus,  the  following  discussions  relating  solutions  to  strategies  were  based  on  only 
responses  to  that  problem.  Fifty-seven  solutions  were  used  in  the  analysis. 

For  the  present  purposes,  identification  of  an  optimal  solution  was  not  necessary.  In  naturalistic 
settings,  finding  a  satisfactory  and  feasible  solution  is  the  goal.  Participants’  solutions  to  the  Bridge 
problem  were  summarized  and  sorted  into  groups  based  on  the  characteristics  of  participants’  final 
proposed  course  of  action.  Two  experimenters  performed  sorts  and  the  discrepancies  between  groupings 
were  resolved  through  discussion.  The  sorting  process  took  the  following  form. 

During  the  sorting  process,  several  different  possible  conceptualizations  for  grouping  solutions 
emerged.  A  nine-group  sort  was  based  on  possible  friendly  forces’  actions  (e.g.,  defend,  fix,  attack,  clear, 
seize,  bypass)  on  possible  objectives  (e.g.,  current  location,  assembly  area,  bridge)  as  they  moved  over 
terrain.  These  basic  groups  were  then  regrouped  by  dimensions  of  the  problem.  Five  groups  were  sorted 
based  on  collapsing  over  the  nine  groups  on  dimensions  of  movement  forward  (e.g.,  defend,  attack. 
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bypass,  assembly  area,  bridge).  Six  groups  were  sorted  based  on  the  primary  focus  of  the  solution  (e.g., 
defend,  bypass,  assembly  area,  bridge,  south  of  the  river,  north  of  the  river).  Four  groups  resulted  from  a 
sort  based  on  possible  perceptions  of  uncertainty  in  the  situation  (e.g.,  how  uncertain  or  unclear  was  the 
information  about  the  enemy,  what  level  of  risk  was  present  in  different  courses  of  action,  importance  of 
assembly  area  to  the  original  mission,  did  the  mission  change). 

The  groups  resulting  from  each  sort  were  evaluated  by  discriminant  analysis  using  the 
importance  ratings  for  all  strategies  as  predictor  variables.  Discriminant  procedures  were  employed  as 
exploratory  analyses  (see  Klecka,  1980;  SAS  Institute,  1988;  Tabachnick  &  Fidell,  1989).  Forward 
method  stepwise  discriminant  analysis  {p  -.25  to  enter  the  equations)  was  first  used  to  identify  a  subset  of 
strategies  that  entered  the  function  for  each  solution  grouping.  For  each  grouping,  the  resulting  strategies 
for  the  equation  were  then  entered  as  the  predictors  in  a  discriminant  analysis.  The  number  of 
misclassified  cases  and  the  proportion  of  variation  explained  were  used  to  compare  solution  groupings. 
The  equation  with  the  fewest  misclassifications  and  which  explained  the  most  variation  (Average  Squared 
Canonical  Correlation)  was  the  function  for  the  four-solution  group,  which  was  based  on  perceptions  of 
the  situation  (see  Table  4). 


Table  4 

Results  of  Discriminant  Classifications  for  Each  Grouping  of  Solutions 


9  groups 

5  groups 

6  groups 
4  groups 


5 

.59 

friendly  mission 

9 

.63 

forward  movement 

4 

.55 

terrain  focus 

0 

.70 

situation  assessment 

Maps  illustrating  each  solution  are  shown  in  Figure  6.  Definitions  for  the  four  solutions  follow. 
The  percent  of  total  solutions  in  each  group  are  given. 


Solutions: 


(1)  Hold  up  and  assume  a  defensive  posture.  The  situation  is  uncertain.  Establish  positions  on 
high  ground.  (5%) 

(2)  Attack  and  occupy  the  assembly  area.  This  area  is  still  the  jumping  off  point  for  tomorrow’s 
mission.  (14%) 

(3)  Attack  and  clear  through  the  assembly  area  and  proceed  to  the  river  and  take  a  position  on 
the  far  side  of  the  bridge.  (58%) 

(4)  Fix  the  enemy  in  the  assembly  area  and  bypass,  use  an  alternate  assembly  area,  or  forget  any 
assembly  area  and  focus  on  the  bridge.  (23%) 

The  four  categories  were  based  on  taking  into  account  the  uncertainties  in  the  evolving  situation, 
confidence  in  the  capacity  to  defeat  the  enemy,  risk  involved  with  continuing  with  the  original  mission 
orders  or  modifying  them,  and  basing  a  solution  on  doing  what  one  knows  how  to  do  rather  than  on  the 
situation. 
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The  relationship  between  strategy  use  and  solutions  was  examined  to  determine  whether  the 
reported  use  of  a  strategy  was  associated  with  a  particular  solution.  Relationships  between  strategies  and 
solutions  could  be  important,  particularly  for  instruction,  if  certain  courses  of  action  were  more  desirable 
on  some  characteristics  than  others.  The  four-group  classification  scheme  for  the  solutions  was  used. 
Results  showed  how  differential  use  of  strategies  was  associated  with  different  solutions. 

The  strategies  that  entered  the  equation  during  the  stepwise  discriminant  analysis  were  entered  as 
indicators  in  a  discriminant  analysis  using  the  four  solution  groups  as  class  variables.  Table  5  shows  the 
strategies  and  coefficients  for  each  solution.  In  discriminant  analysis  the  coefficients  are  estimated  to 
result  in  the  “best”  separation  between  groups  (Norusis/SPSS  Inc.,  1988).  Interpretation  of  the 
coefficients  is  similar  to  that  of  multiple  regression.  Such  interpretation  must  be  made  cautiously  when 
indicators  are  correlated  (Norusis/SPSS  Inc.,  1988).  The  informativeness  and  interpretation  of  the 
coefficient  for  a  particular  variable  is  relative  to  the  other  variables  included  in  the  classification  function. 
Positive  weights  are  highlighted  for  the  15  strategies  that  had  a  range  of  positive  and  negative  weights 
across  solutions.  Seven  strategies  were  positive  indicators  for  all  solutions.  These  are  shown  separately 
on  the  lower  portion  of  the  table. 

Table  5 

Classification  Functions  for  Solutions  by  Strategies 


Strategy 

Stop 

AA 

AA  +  Bridge 

Bypass  +  Bridge 

P26 

P5 

Pll 

P20 

P28 

P15 

C17 

C2 

P23 

P29 

P2 

Cl 

C8 

C12 

C6 

Information  reliability 

Determined  parts  prone  to  flaws 

Broke  into  smaller  problems 

Wrote  down  everything 

Considered  information  relevancy 

Imagined  best  outcomes 

Eliminate  some  options 

Option  met  own  standards 

Used  specific  and  precise  comparisons 
Suspended  judgment 

Thought  of  differences  with  past  experience 
Small  set  of  considerations 

Qualitative  comparisons 

Solution  had  worked  most  often  in  past 

Only  large  disadvantages 

-1.11 

-2.82 

-5.13 

-1.27 

-3.03 

-3.83 

-1.46 

-.13 

-2.14 

-1.52 

-1.86 

-2.92 

-.97 

-1.11 

f’,i56v 

-1,07 

-1.91 

-.93 

-.52 

-2.02 

19  > 

-.94 

-2.51 

-1.52 

-2.42 

-.70 

-.27 

-.03 

-.19 

-.05 

-.98 

-1.52 

-.51 

-2.40 

Positive  indicators  for  all  solutions 

P27 

Information  accuracy 

5.55 

8.29 

4.72 

2.38 

P12 

Big  picture  first 

4.90 

2.69 

1,68 

1.62 

P22 

Identified  assumptions 

3.63 

4.57 

3.04 

■  1.77 

P14 

Kept  set  small 

2.86 

3.50 

1.58 

1.07 

P7 

Problem  as  story 

2.78 

2.96 

.85 

.50 

C15 

Best  option  had  to  meet  required  criteria 

1.34 

2.89 

2.08 

1.06 

C19 

Reexamined  options 

.65 

3.21 

3.14 

2.40 

Coefficients  can  be  used  to  interpret  differences  between  solutions.  (Klecka,  1980).  The  sign  and 
relative  magnitude  of  the  coefficient  represent  how  the  strategy’s  presence  or  absence  was  related  to  that 
particular  solution.  Just  how  these  strategies  relate  to  one’s  experience  is  not  yet  clear,  although  plausible 
hypotheses  were  suggested  by  the  groupings.  One  explanation  is  that  the  “hold  back”  and  the  “bypass” 
solutions  were  both  proposed  by  those  participants  who  considered  the  larger,  more  uncertain  aspects  of 
the  situation  (e.g..  What  is  the  overall  division  situation?  The  enemy  in  the  AA  is  not  my  problem.  Do  I 
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^  0"  the  other  hand,  those  participants  who  became  involved 

m  a^ing  into  or  through  the  AA  seemed  to  become  involved  in  handling  what  they  recognized  as  a 
situation  mvolving  contact  with  an  enemy-one  they  had  been  trained  for  as  part  oSitaiy 

th^  ^  becoming  tmm^d  in  maneuvering  to  thwart  enemy  operations  characterized 

these  ^ups  (e.g^  I  will  attack  from  each  flank  in  a  pincher  movement.  Are  there  enemy  informants  in 
the  village?  That  is  my  designated  area.  I  will  lead  with  my  tanks.)  In  other  words,  they  solved  the 
problem  with  what  they  knew,  rather  than  considering  whether  it  was  the  right  problem  to  solve. 

Pruning  the  Strategy  Set 


Rather  than  rcklting  In  a  smaller  set  of  strategies,  analyses  of  the  data  suggest  several  ways  that 
knowledge  about  the  strategies  could  be  employed.  For  example,  strategies  that  were  positive  indicators 
^ross  all  solutions  (from  Table  5)  should  be  examined  to  find  ways  to  make  them  more  useful  without 
increasing  overall  effort  or  time  demands.  Most  of  the  strategies  used  for  the  sorting  task  have  associated 
strengths  and  weaknesses.  The  frequency  and  magnitude  of  errors  attributable  to  the  weaknesses  of  the 
strategies  being  used  can  be  determined. 


Strategies  friat  were  positive  and  negative  indicators  for  particular  solutions  need  to  be  examined 
for  their  relationships  wifo  other  strategies  (from  Table  5).  For  instance,  people  who  Imagineci  the  best 
outcomes  (PI  5)  did  not  appear  to  Imagine  the  worst  outcomes  ( P16).  At  least,  it  did  not  enter  the 
functions  for  die  solutions.  However,  Imagined  parts  of  the  plan  that  would  be  prone  to  flaws  (P5)  was 
used  by  some.  Those  people  proposed  different  solutions  than  did  the  people  who  did  not  use  it 

Success  using  the  “best  outcome”  method  depends  largely  on  domain  knowledge  and  the  ability 
to  distin^ish  relevant  from  Irrelevant  information.  It  is  an  efficient  and  quick  process.  However,  those 
who  use  it  might  also  fail  to  search  for  counter-examples  or  foil  to  understand  implications  of  the ' 
situation.  Identifying  these  failure  points  to  people  who  tend  to  naturally  adopt  a  “best  outcomes” 
strategy  would  probably  enhance  solution  quality.  To  dwell  only  on  success  without  considering  the 
perils  of  the  situation  impacts  contingency  plans,  an  important  component  of  planning. 

Similarly,  strategies  such  as  (P28)  Identified  relevant  inforTnation  and  (C 1 2)  7%e  solution  that 
had  worked  most  often  m  the  past  was  chosen  merit  examination.  Results  shown  in  Table  5  illustrate  that 
the  use  of  these  strategies  was  associated  with  different  proposed  solutions 

Strategies  which  were  used  infrequently  but  which  have  the  potential  for  big  improvements 
should  al»  be  included.  Powerful  strategies  which  were  important  to  only  a  few  people  might  have  been 
obscured  in  group  mean  ratings  of  importance.  For  example,  past  studies  have  found  that  the  ability  to 
pick  out  diagnostic  information  and  to  use  one’s  experience  and  knowledge  to  find  solutions  are 
hallmarks  of  skilled  problem  solvers  (Abernathy  &  Hamm,  1995;  Jacavone  &  Dostal,  1992). 

Lines  of  Continuing  Research 

Generalizing  findlncs  from  problems  Findings  from  solutions  to  the  Bridge  problem  will  be 
compared  with  solutions  for  the  Hostage  scenario.  This  will  elaborate  understandings  of  the  relationship 
of  strategies  and  solutions  relative  to  different  types  of  problems.  Some  participants  described  the  Bridge 
scenario  as  having  more  long-range  consequences  and  xmfolding  into  future  problems,  whereas  the 

Hostage  problem  was  a  single  event  that  would  end  when  the  friendly  forces  egressed  the  compound  and 
renmied  the  hostages  to  friendly  territory. 
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Dealing  with  uncertainty  Perception  of  uncertainties  in  a  situation  might  influence  strategy  use. 
For  example,  some  participants  saw  the  uncertainty  in  the  Bridge  problem  in  determining  the  most 
important  thing  to  do  (e.g.,  clear  out  the  enemy  or  take  the  bridge).  On  the  other  hand,  for  some 
participants  the  goal  in  the  Hostage  problem  was  clear:  rescue  the  ambassador  and  his  family.  Other 
participants  found  the  goal  of  the  Hostage  problem  to  be  less  clear.  Was  it  to  cause  confusion  in  the 
minds  of  the  captors,  to  keep  the  ambassador  and  the  family  from  harm,  to  retain  diplomatic  sovereignty 
of  which  the  ambassador  was  a  symbol,  or  to  send  a  message  to  other  terrorists? 

Strategies  and  approaches  for  dealing  with  uncertainty  need  further  identification.  People  who 
have  solved  problems  in  uncertain  situations  might  have  developed  efficient  strategies  that  have  gone 
unnoted,  either  because  of  the  scarcity  of  this  type  of  research  or  because  the  responses  of  a  few 
individuals  were  unnoticed  in  group  analyses. 

Identify  other  influences  Despite  the  effort  to  be  comprehensive,  it  is  possible  that  there  are  other 
influences  on  problem  solving  which  were  not  captured  by  the  set  of  strategies  used  for  this  study. 

Factors  that  might  have  important  effects  on  problem  solving,  such  as  risk ,  personal  values,  and  personal 
style,  were  not  included  in  the  present  set.  It  might  be  that  these  types  of  variables  have  more 
significance  in  uncertain,  ill-defined  situations  than  supposed.  Based  on  participants’  comments  from  the 
transcripts,  these  factors  should  also  be  more  specifically  identified. 

Instantiating  strategies  Understanding  how  strategies  interact  with  individual  differences,  such  as 
prior  knowledge  and  experiences,  would  be  useful.  Participants’  comments  would  be  examined 
concurrently  with  the  card  sort  information  to  determine  whether  the  use  of  the  same  strategies  lead  to 
different  solutions  and,  if  so,  why  they  did. 

One  possibility  is  that  strategies  interact  with  prior  knowledge  to  generate  instantiations  of  the 
current  situation.  For  example,  although  two  different  participants  rated  Identified  a  specific  goal  as  most 
important  to  their  problem  solving  process,  they  may  have  identified  different  goals,  which  might  result 
in  different  solutions.  One  participant  who  opted  to  take  the  assembly  area  said:  “...you  have  to  take 
these  guys  down  so  that  you  can  accomplish  your  mission.  So,  I  really  have  two  missions  but  right  [now] 
my  specific  goal  is  I  have  to  get  into  that  assembly  area,  I  have  to  take  down  enemy  forces  in  that  area 
before  I  can  go  and  do  my  next  goal  which  is  to  cross  the  river  at  0400.”  In  contrast,  a  participant  who 
chose  to  bypass  the  assembly  area  in  favor  of  focusing  on  the  bridge  said:  “...[t]he  bridge  was  the 
objective  in  this  case.  Normally,  the  enemy  is  the  objective  and  you  want  to  destroy  the  enemy,  but  in 
this  case,  for  the  division  to  move  through  tomorrow,  the  bridge  becomes  more  important.. .is  the 
objective.”  Thus,  although  both  participants  said  that  they  used  the  same  strategy,  the  outcomes  of  their 
thinking  processes  (their  solutions)  were  different  due  to  content  differences. 

It  may  be  that  individual  strategies,  such  as  (Cl  2)  Used  the  solution  that  had  worked  most  often 
in  the  past,  can  be  leveraged  when  combined  with  strategies  to  select  appropriate  or  relevant  knowledge 
and  information.  Examination  of  participants’  comments  will  be  helpful  in  clarifying  this  relationship. 

Verify  metacognitive  knowledge  An  assumption  in  this  research  and  others  was  that,  when 
prompted,  participants  can  provide  useful  information  by  being  aware  of  and  reflecting  on  their  own 
thinking.  However,  the  limitations  and  problems  of  using  interview  data  have  been  discussed  elsewhere 
in  detail  (Ericsson  &  Simon,  1993;  Rowe,  1985). 

If  card  sorts  do  not  faithfully  represent  participants’  thinking  processes  or  if  participants  were  not 
able  to  fully  translate  their  thinking  processes  to  the  cards’  definitions,  then  findings  from  the  card  sorting 
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task  can  be  supplemented  by  the  transcripts  from  the  interviews.  When  strategies  are  examined  relative 
to  content,  it  may  be  discovered  that  participants’  card  sort  data  lacked  correspondence  with  the  contents 
of  their  interviews.  For  example,  a  participant  might  have  rated  (PI  2^  Looked  at  the  big  picture  before 
the  details  as  more  important  in  his  thinking  than  (P13)  Considered  the  details  before  the  big  picture,  but 
verbalized  a  focus  on  the  details  first  in  his  conversation  about  the  problem.  If  this  was  found  to  be  the 
case,  strategies  could  be  assigned  to  include  what  they  did  instead  of  what  they  thought  they  did. 

Training  metacognitive  knowledge  and  skill  Many  of  the  participants  commented  after  the 
interviews  that  they  enjoyed  and  were  challenged  by  “thinking  about  what  they  were  thinking.”  Some 
made  comments  like  “I  didn’t  use  this  strategy  but  maybe  I  should  have.”  One  participant  noted  that  he 
hadn’t  really  thought  about  the  “big  picture”  and  where  friendly  forces  would  be  on  his  flanks  as  he 
attacked  in  zone.  He  realized  that  he  had  earlier  made  assumptions  about  the  situation  which  might  have 
been  unwarranted  and  dangerous  had  he  been  in  actual  combat. 

Comments  such  as  these  and  the  preliminary  results  of  a  training  module  (the  results  of  which 
were  not  included  in  the  present  paper)  related  to  one  of  the  strategies  suggest  that  instruction  in  thinking 
skills  might  be  a  fruitful  avenue  to  pursue. 


Conclusion 


Approaching  Problems 

The  analyses  demonstrated  that  the  participants  adopted  a  variety  of  approaches  to  solve 
problems  rather  than  adhering  to  one  method.  Participants’  ratings  of  their  approaches  support  the  notion 
that  combinations  of  approaches  were  used.  Thus,  current  “schoolhouse”  emphasis  on  procedural  and 
analytic  methods  neglects  other  approaches  that  are  reportedly  being  used  on  the  job.  This  observation 
was  supported  by  several  participants’  comments  to  the  effect  that  “We  were  taught  it  in  the  course,  but 
we  go  home  and  we  don’t  use  it  much.”  The  strengths  and  weaknesses  of  these  and  other  approaches 
should  be  pointed  out,  explained,  and  supported. 

Problem  Strategies 

Examining  how  strategies  were  used  and  how  they  were  combined  illustrated  that  differential  use 
of  strategies  was  associated  with  different  proposed  solutions.  By  making  problem  solvers  aware  of  the 
strategies  that  they  do  use  and  the  points  where  the  strategies  are  fallible,  natural  tendencies  could  be 
enhanced  and  potential  for  large  errors  diminished.  The  results  of  these  analyses  provide  evidence  for 
selecting  strategies  for  further  examination  with  the  intention  of  enhancement  and  support. 

Future  Directions 

One  approach  to  improving  problem  solving  in  complex  and  dynamic  problem  situations  would 
be  to  increase  knowledge  about  the  various  problem  situations  that  one  might  encounter.  The  problem 
with  this  perspective  is  that  dynamic  and  ill-defined  problems  are  novel  cases  by  nature.  It  would  be 
impossible  to  predict  all  of  the  relevant  attributes  and  contingencies  to  be  captured  and  trained  for  a  set  of 
ail  possible  future  situations.  Therefore,  since  all  knowledge  and  information  can’t  be  known  at  any  point 
in  time  and  all  possible  cases  can’t  be  identified,  a  different  plan  would  be  to  focus  on  methods  to 
increase  problem  solvers’  abilities  to  leverage  their  knowledge  and  situational  information  to  meet  the 
demands  of  particular  circumstances  in  which  they  find  themselves. 
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Thus,  one  potential  way  to  increase  flexible  problem  solving  ability  might  be  through  raising 
awareness  about  the  strategies  that  one  uses  and  about  other  strategies  that  could  potentially  be  useful. 
However,  situations  for  training  these  types  of  metacognitive  skills  should  be  chosen  carefully.  The 
ultimate  purpose  of  elaborating  one’s  set  of  strategies  is  so  that  they  can  be  used  in  time-constrained 
situations  with  little  effort.  Therefore,  the  effortful  processing  about  the  strategies,  which  might  initially 
degrade  performance  during  practice,  should  be  enabled  before  one  needs  to  use  them,  much  like  the 
proactive  preplanning  work  done  by  effective  flight  crews  (Wickens,  1996). 

Benefits 

Understandings  about  how  problem  solvers  actually  solve  problems  could  enhance  training  by 
identifying  ways  that  successful  problem  solvers  respond  to  situations.  These  techniques  could  be 
demonstrated  to  the  less  experienced. 

Training  time  could  be  exploited  to  amplify  existing  strengths  rather  than  to  enforce  unnatural 
methods.  For  example,  one  participant  said  that  he  used  solutions  that  had  worked  most  often  in  the  past, 
always  spliting  teams  in  half  when  attacking  buildings.  Based  on  his  past  negative  experiences  when  fire 
support  had  been  missing,  he  believed  that  forces  clearing  buildings  should  always  have  covering  and 
suppressing  fire  support.  He  relied  on  his  knowledge  about  similar  situations  to  plan  his  course  of  action: 
“If  it  worked  in  the  past  there  is  no  reason  why  it  won’t  work  again  in  the  same  situation  or  similar 
situations.”  He  did  not  pause  in  his  problem  solving  process  to  decide  whether  to  have  supporting  fire 
and  how  much.  He  relied  on  a  strategy  using  frequency  of  past  experiences  and  matching  processes  to 
match  one’s  knowledge  with  the  current  situation.  However,  effective  use  of  this  strategy  depends  on 
finding  a  good  match  between  experiences  and  the  current  situation.  Thus,  how  the  problem  solver 
defines  the  problem  and  identifies  relevant  information  influences  the  success  of  this  strategy. 

Recommendations 

A  basis  for  instruction  and  training  can  be  established  by  understanding  how  successful  problem 
solvers  differ  from  the  less  experienced  in  the  ways  that  they  go  about  solving  problems.  Including 
analysis  of  the  strategies  and  solutions  to  the  Hostage  problem  to  those  of  the  Bridge  problem  would 
broaden  the  findings  to  multiple  scenarios.  Examining  similarities  within  subgroups  of  participants 
would  uncover  whether  efficient  or  new  strategies  went  unnoted  because  they  were  only  used  by  a  few 
people.  Strategies  and  approaches  that  may  be  efficient  and  have  been  unnoted  should  be  catalogued. 
Understandings  of  other  individual  variables  that  influence  problem  solving,  such  as  personal  values  and 
reactions  to  uncertainty  should  be  developed.  Ongoing  research  findings  and  issues  should  be 
communicated  to  practitioners  so  that  they  can  begin  to  capitalize  on  the  strengths  of  naturally  occurring 
problem  solving  approaches  and  strategies. 
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Appendix  A 

Definitions  of  Approaches 

Participants  assigned  points  to  the  following  four  definitions  when  identifying  their  general  and 
problem-specific  approaches.  The  definitions  were  read  without  identifying  labels. 

Analytic:  Always  generate  options  systematically,  identify  criteria  for  evaluating  these  options, 
assign  weights  to  the  evaluation  criteria,  rate  each  option  on  each  criterion  and  tabulate  the  scores 
to  find  the  best  option.  Use  probability  and  utility  to  evaluate  alternatives  and  to  calculate 
possible  outcomes.  The  alternative  with  the  highest  outcome  is  selected. 

Procedural:  Clarify  the  situation  and  gather  information  without  making  judgments  about  it. 
Identify  several  alternative  solutions.  Do  a  complete  analysis  of  the  merit  of  each  alternative 
before  comparing  them  to  one  another.  Select  the  best  alternative  based  on  the  results  of  the 
analysis.  Implement  the  solution  If  the  chosen  alternative  does  not  work,  begin  the  process  of 
selecting  alternatives  again. 

Recognition:  The  problem  is  recognized  as  familiar.  If  the  situation  is  very  familiar,  then  a 
familiar  solution  may  be  recognized.  Consequences  of  using  the  usual  solution  are  imagined  to 
identify  pitfalls.  If  unworkable,  then  an  alternative  is  developed  based  on  what  goals  are  feasible, 
what  information  is  important,  what  to  expect  next,  and  what  actions  are  typical  in  this  situation. 

Dominance:  A  promising  alternative  is  identified  and  the  others  that  have  no  chance  of  becoming 
the  best  overall  are  discarded.  Determine  whether  the  promising  alternative  has  any  disadvantage 
compared  to  other  alternatives  or  to  some  other  criterion.  If  it  doesn’t  have  any  disadvantages  it 
is  chosen.  If  it  does,  then  information  about  the  alternative’s  advantages  and  disadvantages  is 
reconsidered.  Upon  reconsideration,  that  alternative  is  discarded  and  the  process  repeated. 
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Appendix  B 
List  of  Strategies 


Processing  Strategies 

Thought  of  similarities  between  the  information 
in  this  problem  and  what  was  already  known 
from  past  experience. 

Thought  of  differences  between  the  information 
in  this  problem  and  what  was  already  known 
from  past  experience. 

Deconflict  information. 

Thought  in  terms  of  if-then  thoughts.  For 
example,  if  the  weather  is  bad  then  an  alternate 
route  is  needed. 

Determined  parts  of  the  plan  that  would  be 
prone  to  flaws. 

Considered  what  information  was  missing  and 
its  implications. 

Looked  at  the  problem  in  terms  of  a  story  to 
develop  a  fuller  understanding. 

Considered  the  solution  in  terms  of  a  story  to 
develop  a  fuller  understanding. 

Restated  the  problem  by  visualizing  or  drawing. 

Restated  the  problem  in  different  terms  by 
looking  at  it  as  a  different  type  of  problem.  For 
example,  an  offensive  problem  was  relooked  as 
a  defensive  problem. 

Broke  the  problem  into  smaller  problems. 

Looked  at  the  big  picture  before  the  details. 

Considered  details  before  the  big  picture. 

Kept  the  set  of  things  to  think  about  as  small  as 
possible. 

Imagined  the  best  outcomes. 


Imagined  the  worst  outcomes. 

Spent  time  considering  various  perspectives  on 
the  situation. 

Identified  a  specific  goal. 

Identified  problems  in  accomplishing  the  goals. 

Wrote  down  everything  that  I  knew  about  the 
problem. 

Identified  facts. 

Identified  assumptions. 

Used  specific  and  precise  comparisons. 

Used  general  and  approximate  comparisons. 

Identified  information  that  was  unusual. 

Considered  the  reliability  of  information. 

Considered  the  accuracy  of  information. 

Considered  the  relevancy  of  information. 

Suspended  judgment  about  possible  alternatives 
until  all  the  information  was  examined. 
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Choice  Strategies 

The  best  option  had  to  meet  certain  required 
criteria. 

The  best  option  should  have  met  certain  desired 
criteria. 

Some  options  were  eliminated  before  comparing 
options. 

More  than  one  option  was  found  to  be 
acceptable. 

Acceptable  options  were  reexamined  to  see  if 
one  had  more  important  characteristics  than 
another. 

An  option  was  selected  if  it  met  my  own  internal 
standards. 

An  option  was  selected  if  it  met  standards 
specified  by  others. 

Chose  the  option  that  had  at  least  one  important 
characteristic. 

Chose  the  option  that  was  better  than  all  the 
others  on  one  important  characteristic. 

An  option's  disadvantage  was  considered 
important  only  when  it  was  a  large  disadvantage. 

Trade-offs  were  made  between  an  option’s 
advantages  and  disadvantages. 

Options  were  judged  qualitatively,  for  e.xample, 
good,  bad,  acceptable,  etc. 

Options  were  evaluated  using  quantitative 
(numerical)  assessments. 

Used  a  screening  technique  to  select  options  for 
further  consideration. 

The  option  that  had  occurred  most  often  in  the 
past  was  chosen. 


The  solution  that  had  worked  most  often  in  the 
past  was  chosen. 

The  option  that  was  most  attractive  on  the  most 
important  characteristic  was  chosen. 

If  the  proposed  solution  was  acceptable,  no 
other  solutions  were  considered. 

Kept  the  set  of  things  to  think  about  as  small  as 
possible.  (This  strategy  was  intentionally  used 
in  both  sets.) 
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